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ABSTRACT - 

„ This paper presenile a- means of evaluatiag .answers to 

comprehension guestions by analyzing the oral language used in the 

answers to the guestions according to i;he average number \<>± wor.ds per 

/communication unit, a measure of linguistic an.d cognitive- growth, in 
order tq determine the ..effectiveness of comprehension guteslAions asked 
eft two cognitive levels to stimulate the higher cognitive^ptocesses. 
In two research studies. students were asked. questions at twfa levels 
of comprehension. Their respon^^s were "tape recorded, analysed, and. 
compared. It .was' foun,a that comprehension guestions aske^ at the V 
higher cognitive level elicited answers that were signi:fiicAntly / . 

. longer than the answers to the questions asked at the loW^r 

comprehension level. The Study determined that the effectiveness /)f 
comprehension guestions to stimulate higher cognitive ptxicesses, can 
be evaluated by -analyzing the language cTiildren use in answering t^e 
questions according to the number of words of the communication unit. 
(Author/IL) ■ ' 
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i> ^ ■ • 

Purpose , , . 

■ . ' . - '■■/••*'. 

Much has been written about instp:ictioaal qu^stions^ 
about the effect of" questions upon student thinking, an,^, 
about the need to increase the percentage of cOrapreheAsion 
questions asked at higher cognitive levels. This tesearch 
presents a means orf evaluating answers to eomprehens^ien 
questions, in reading", listening, visual perception, and 
the related content arenas, art avenue of research that has 
^ only begun to be investigated. ' ' y . ^ 

Background ZnforTnat ion ^ 

A xm^or purpose of education is to enhance childre'n's 
thinkiftg* Piaget (4) viewed a child's verbal accommodation 
to Jfearning experi^ences as having a^asti^ng effect only when 
affected the means of organising one' s experiences . The 
/role: of the question in the learning process" was found to be 
significant in the otganizatidn of experiences. Piaget 's 
research relies heavily upon the use of questions in tit^ 
fartiQUS metho'de^'clinique. . 

Taba (3), basing research heavily upon Piaget, found 

classroom questions to have an enormous influence' on the'* 

critical thinking of students. Taba found that through 

systematic instruction b^sed upon questioning, the^teacher 

•' ■ ■ , • • 

• could stimulate higher cognitive processes^ of children. 

Cognitive processes ha^^^e been related to the level 

of questions. Ruddell (5) explained that questions asked 

at interpretive and applicative levels stitnula^e higher 
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cognitive processes €h?in factual questiortSt Factual 
questions merely /nvolve experience plus the r.ecall of 
immediate information .for the answers, whereas answers 
to 'interpretive^ questions involve the modification of 
the content bj^' analysis/- reconstructioHf' or inference of 

• relationships. Answers to applicative questions involve 
creative prolDlem solving, Ruddell viewed questitmtng 

>' as a valuable instructional tt)ol to develop J^e cognitive 
al])i,lity in reading and listening., 

Lowery (3) , stressing the 'importance, of 'the cognitive 
levels, of instructional questionsr categorized questions . 
into two intellectrual levels, narrow questions requiring 

* 

certain "correct^' answers with planned, convergent outcomes, 
and. broad questions requiring divergent or unplanned answer's 
with at/ least several different acceptable response^ in- 
cluding affective; evaluations. > 

Although much hi^s been written about the classifica- 
tion of comprehension questions into various cognitive 
levels, and about how the levels of questions affect the' 
copnitive processes, few researchers have explored the , 

0ffect|!veness of questions to elicit answers indicative 

' I I ■ ■ ° ■ 

1 oB the use of hi^gher cognitive processes. Little has 
\ been written about the evaluation of answers to compre- 

r ■ " ' ■ 

Ihension questions, or about the relationsh,ip between the 
\level of questions asked and the cognitive processes in- 
^'volVed in the answers to the questions; ^ 
In evaluating the answers to comprehension questions, 
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Gair (1) polluted out that together witb the use of ti^^ ' - 
data to defeind the respoiise, it would reasong^ble to 
;fexpect a correlation between the responsie lengtht' and its 
(Quality. * ' ^,>T} 

In evaluat:ing answers to comprehension questions 
one notes that the language used in the answers, can pro- 
vide evidence of the relationship between the levels of 
questions ask^d and the cognitive processes involved in 
the answers. Researcft in language has shown a correlation 
between the length of children* s grammatical structui?es • 
and their level of cognitive development, ^ 

Noting the relationship of sentence s.trudture and ' 
Piaget's stages of cognitivjg development, Sinclair-de- 
Zwart (6) found linguistically higher order grammatidal 
structures among children who had reached higher opera- 



tional srtages of cognitive development. When Asked to 
describe the differences between two objects, children ^ . 
ifi the preoperational stage, singled out the differences 
in short sentences, such as, "This one is long; This ane 
is short;" whereas the children ^o had operationally 
mastered conservation 6f quantity expressed^the double 
difference in one structurally more complex statement, 
"This one is longer and thinner than* the other one," 

From longitudinal language research, Loban (2) found 
that among school children there is a posxtive correlation 
between the advance in grades and the increasing average ^ 
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word-length of the communication unit, the graminatical 

independent clause and its modifiers • During the first 

seven years of school, children, as they advance in cog 

hitive development, produce more communication units and. 

increase the average number of words in the communication 
♦ 

units each year. Loban '(2) stressed that the average 
number of words per communication unit is a measure of 
significance based on evidence that a high average of ^ 

. words per communication' unit was coupled with increased 
complexity of grammatical structures. 

' Guided by research indicating that the average length 
of the communication unit of children increase .in length 
as they advance to higher stages of cognitive development, 

'and that th6 cognitive levels of comprehension .quest ions 
affect the .thought processes of children. Smith (7) hy- 
pothesized that the language used by children in answer 
to comprehension questions. asked at a higher cognitive 
level would result in significantly long-er average com.muni- 
•cation units thanNvould the answers to compfehenslon ques- , 
tibng asked at a loVer cognitive level. 

Method of Research A ^ • • • 

In the first of two research studies, Smith (7) made, 
an investigation of the oral language of children in answers 
to questions asked at the factual level, a lower cognitive 
. level, and the interpretive level, a higher cognitive level, 
as part of doctoral dissertation research. . • • 
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Smith (7) selected thirty subjects, from each grade 

two .and grade four in order to compare two stages of cog- " 

nitive dievelopmen't , the preoperational period represented 

by grade-two ^subjects, and/the concrete operations period i 

represented by grade-four sjubjects. - The sijcty subjects 

were chosen at random from one California elementary rschdol 

fepresentihg " predominate ty ttie lower and^ middle Trias s C au-~" 

# ' ■». - » . 

casian socio-economic level. 

Each Individually interviewed subject was asked four 
factual level questions and four interpretive leVel ques- 
tidns aboiit each of -three, stimuli. Each subject was pre- 
sented with, three separate stirayli, a reading stimulus 
consisting of a story which the subject read; a listening 
stimulus consisting of a story which was i?ead to the sub- 
^ecti by the" examiner J and a multiple-picture stimulus con- 
sisting of three pictures. After the presentation* of .each 
stinialus, four factual and four interpretive queiations 
'*w^re asked. The factual', the lower cognitive level ques- 
tions, required the subject to utilize experience plus 
nj^mory fiDr the answers^ The interpretive; the higher c6g- 
nit^ive level questions, involved the modification of 'the 
content of the data by anal:ysls, ,redohsti:xiction, or in-' 
ferehce of relationships.. - " = ' . 

The follqwihg are e^tamples of the questions and ' 
answers! . . 

Factual Questioni Who had a walletV 

Answer! . a lady/ 
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Vadtual .Question! JWhat happened to the wallet? 
Answer! - ' \ ehe left it at the store/ \. 

Inteirpretive' Questions r What do you suppose the 
; lady will do with th^ wallet? 
Answer! give it back to the lady/- 
interpretive Questiotit When the lady knows her 

wallet . k€r los^tr what might she da? 
Answer! go back to the store and get it/ 

The tape recorded language ysed by the subjects in their 
answers to the questions was, transcribed and analyzed 
-according to the average nutoer of words per coranmnica- 
tlon unit" usfed in answers ta the questions. 

The communication unit is a linguistic unit that can- 
not be further' divided without loss of its meaning. / It 
consists of a' grammatically independent clause with any 
of its modifiers, and includes no more than one such clause 
The sentence, "I see a girl with a dog," consists of one 
communicp^tion unit.. The jneaning would be lost if the 
sentence were segmented into smaller units, "I see a girl" 
/*with a dog." The sentence, "I see a 'girl and I see^a dog, 
consists of I'two; comraunicaj:ion units since tlte sentence .con- 
sj-sts of, t.^ infiependent "grammatical structures, "I see a 
girl," "and I see a dog." However, the sentence, "I see "a 
girl and a dOg." consists of only one communication unit 
since- it ^cannot be divided- into two meaningful gramnjatical 
structures. ° . , 
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Results of Research* A ^ , ' 

' , — ^ «■ ° • 

Smith mad4 comparisons of the average lengths of the 
communic^ition units elicited in the oral ^language of the. 
subjects of both grades two and four in. answe^rs t6;the 
factual and to the interpretive questions. The research 
findings showed that in answering interpretive comprehend 
sion questions, theS subjects of grade four responded by 
using signifdcantly ^longer communication units than did 
the subjects of grade two, as calculated" by the two- sample 
t-test statistical, analysis. Subjects in grade four used , 
ail average of 9.84 words per communication unit; , compared 
with,.?. 27 words used by the subjects of grade two. (See. 
Table 1.) ' • ' . 

A, ■ ^ ' 

. . Table I- • ' 

ORAL LANGUAGE ^ICITED IN ANSI^ilRING QUESTIONS 

(Comparing Two Levels of Questions) 

Grade Average Words per Communication Unit tVva-lue N 
______ Interpretive Q. Factual Question _ 

2 7.27 3.56'^ 10.77* - 30 

4 . 9.84 " 3.74 12.0^* 30 

?f Significant at .001 level. 

In answer's to cpmprehenstion questions asked at. the 
interpretive level compared to the questions asked- at the^ 
fabtual level the subjects in both grades r6si3ond€d in 
significantly longer average communication units to the- 
comprehension. questions asked at the interpretive level 
as calculat^ed by the matched pairs t-test. Ln answer tov 

*. 9 : 
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the factual questions, however, the' subjects ir| grade four 
did not. respond at a higher , cognitive level by rusing sig- 
nif icant ly ' longer average communication units than the- 
subjects .of grade two. The average length of the responses 
for grade-four subjects was only 3.74 words and 3.56 words 
for the subjects of grade to in answers to the factual 
questions. (See Table L) ^ . 

Using the average length of the communication unit as 
a measure of cognitive and linguistic growth. Smith viewed . 
the interpretive comprehension questions, eliciting signifi- 
cantly longer answers, to' be more effective in stimulating - 
higher cognitive processes. "The research results indicated 
that the type, of questions asked influenced the -cognitive 
levels of the subjects of grade two and grade four. Ques- 
tions at the interpretive level . prompted answers that were 
two to. three times longer than the answers to the factual 
questions which indicated that the interpretive comprehension 
questions were more effective in stimulBting higher cogni- 
tive processes for the ^ub jebts- of both grades - measur&ble 
by dete-rmining the average number of words per communication 
• unit elicited in the answers to the questions.- - . 



■I ■ 



Method of Research B 

Using the i^verage length of t|e commi|nication unit for. 
.language growth measures,' Smith analyzed the oral language 
used by -students in answers to questions designed by teachers 
participating in" a graduate level teacher education project 
implemented through the cooperation of the School of. 
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^ • i- ^ f - .. • • ■ 

Education of the University cJf California, Berkeley, and 
the Lawrence Hall of Science.'^ Both elementary /and secondary 
teachers participated in a^gradUate tr'aining program aimed 

improving comprehension questions in their classroom ' r- 
instructlDn in reading as well, as %n the content areas 
The-'teachers designed varidus instructional questions '^at 
two cognitive •le\^ls, the narrow level and' the -broad leVel. 

The 'narro^' questions consisted of direct inf oirmatton ques- • 

« * ' •■' ' 

'tbon's requiririg the^ recall or the recogoition of i-nfarma- 

^ ' , " . . >- 

tion, and facuging'^questions requiring the students to 

'* ■■■ c . ■ ' ' ^ 

develop a particular idea or answer by leading him toward. • 

; . <■ . ' - • - - 

it through clUes as 'to- what the -answer is or- the appropriate' 

methcM of ^lyiainfing- the answer. . . •' ^ ' 

The fallowing are exarrtples of »the questions and , . 

answers'! * ^ . ' ' \ - 

-. , • Narrow Question.! .What did tljie little- girl' find ih ' S 
•• ■ . 'the wbodst ■ ' ^ ' ' . 

Answer!^ a gynniwolf/ ' . - ' — - 

. BroadvQuestion! ■r\?h^t does the gunn i wo lf*» remind you of? 
Answer!' he looks like 'a. real mad woif that's going 
' . 'to ©at", something up/ • 

The tfro ad questions allowed for several acceptable answers 
related- to it. The broad questions included open-ended 
questions allowing the student to explore freely, without 
restrictions and with only minimal, guidance toward develop- 
ing the answer^ and valuing questions, asking for an affective 
evaluation ot. an explanation of the criteria used 'in, making 



' . . ^ Evaluating hnswery^ 'Q^^0Zion3 • 11 

/' an 'evaiaation. . • x .■ 

Xape.^recofdings of ■ the ciassroofli questions were 
^ ^llecte^ fr6m twenty teachers- as they asked questions 
jdurins individual or group leqsons. Smith transcribed 
■& th6 "^ape" recordings and analyzed the language used by 
' '"'the stude.Ots in their answers, to the "^est ions asked 
at^he ^rt^J^ow and the bi^oad levels of comprehension 
according to the avera^ number of wor?is per communica- 
' tiort unit. ' 



Results of. Research' B . - ^ 

• The analysis of t^ oral language used , by students , ^ 
of ."^He/^'wenty classes from the '^rarious grades in answers , 
to. 1,511, q'uestions showed that the Werage number of words" 
used in/the answers to^ the broad questions at the higher 
leyel of comprehension was 4'. 46 *7ords in°'answer to- 568 
questi-ons compared to an average of °3 .13 words -in answers 
V> 943 narrow questions at th§ lower Cognitive lev^l of 
comprehension. (See Table 2.) Again, th^" questions- aske^ ■ 
at the higher cognitive level elicited responses t|^t. were 
considerably lorYger than the anst^ers to -the quest ions, at the 

-a ■ ^ . ■ t 

lower cognitive level. 

■ . ■ ' , Table 2 ■' ^ 
>RAL LANGUAGE ELICITED IN ANSl^ERING QUESTIONS - , 

Cognitive Level Number of Average Words per 

of Question ^ Questioris . Communication Unit 

Narrow (Lower) .983 « 3.1'3 ?\ 

• . " ■ ^ 

Broad (Higher) 568 '" 4.46 
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J Altbough the higher cognitive questions elicited longer 
average responses than the lower ^cognitive. quest ions, in 
examining the responses to specific questions not all 

brd^d^^questtons were consistent in eliciting longer responses. 

. ■■ ' ' .. 'V ■ ■ , . ^ ^ ■• . . 

For example, the atnswers to broajd'^yaluing questions concerned 

aboW pB»sondl fee liTigs usually el-ic it ed brief answers in- ^ 

dicative of lower cognitiv.e level responses. For example, 

in answdMp the valuing-^question, "How would you feel?" 

the brief response might.be, "gladi" "bad," "terrible," . . 

"s^d," or "happy." It is necessary to look at the answers 

to specific questions of each category in 6rder to determine 

* ' ■■ ... 

whether the answer is reflecting the expected cognitive level 
responses. , ; * 

Conclusions and Imolications for Education . 

The answers to comprehension questions can be evaluated 
by measuring the number of words used in the answers to the . 
questions. The effectiveness o£ comprehension questions to 
elicit ansyers indicative of higher cognitive processes can 
be determined by the analysis of the language used in the 
answers according to the number of words in the communication 
units used in the answers. Based upori the evidence that the 
communication unit is a measure of lihguistici and cognitive • 
growth, it IS viewed that the higher cognitive questions 
eliciting answers in significantly longer average communica- 
tion units are more effective in enhancing the use of higher 
cognitive TDrocesseS and in improving comprehension. 

Comprehension in reading, listening, and visua-1 percep- 
tion, as well as' comprehension related to the various content 
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areas i- can be improved Uy. -de signing and including in the . 
instructional programs a greater percent age of questions 
that ^elicit answer? indicative of the.iise of higher cogni- 
t.ive processes and by ttie evaluation of the answers to the 
'questions in order to determine wAether the questions ap 
actually eliciting ahewers at the cognitive levels for 
.'^hich they were; designed. . , 
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